Scanning apparatus and method

ABSTRACT

A scanning apparatus and method for generating computer models of three-dimensional objects comprising means for scanning the object to capture data from a plurality of points on the surface of the object so that the scanning means may capture data from two or more points simultaneously, sensing the position of the scanning means, generating intermediate data structures from the data, combining intermediate data structures to provide the model; display, and manually operating the scanning apparatus. The signal generated is structured light in the form of a stripe or an area from illumination sources such as a laser diode or bulbs which enable data for the position and color of the surface to be determined. The object may be on a turntable and may be viewed in real time as rendered polygons on a monitor as the object is scanned.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a broadening reissue of U.S. Pat. No. 7,313,264,issued Dec. 25, 2007, which is a continuation of U.S. patent applicationSer. No. 09/000,215, filed on May 26, 1998, now U.S. Pat. No. 6,611,617,which is a 371 of international application PCT/GB96/01868, filed Jul.25, 1996, which in turn claims the benefit of British Application No.9515311.0, filed Jul. 26, 1995.

FIELD OF THE INVENTION

This invention relates to an apparatus and method for scanning athree-dimensional object.

BACKGROUND OF THE INVENTION

Real-world, three-dimensional objects, whether with natural form (e.g.,geographical, plant, human, or animal-like) or man-imagined form (e.g.,sculptures, reliefs, cars, boats, planes, or consumer products) aredifficult to scan. This is because of features such as rapidly varyingsurface normals and surfaces for which a line of sight is difficultbecause it is partially obscured by other parts of the object.

Scanning machines—also known as digitizing machines—for scanning objectsor parts of objects can be categorized into two types—computernumerically controlled (CNC) and manually operated. A scanning machineincludes a unit that contains a sensing means commonly referred to as aprobe.

Objects or parts of objects can be scanned on CNC scanning machines witha number of computer numerically controlled (CNC) linear and rotatingmotor-driven axes. Different CNC machines can move/reorient the probe orthe object—or both—by a combination of translation and rotation aboutthese axes. Different machine designs are suited to different classes ofobjects. Probes can be temporarily or permanently attached to most typesof CNC machine tool or CNC coordinate measuring machine that can then beused for scanning. As examples, small and simple three-axis CNC millingmachines may be used or large, complex five-axis machines may be used.The points captured by CNC machines are usually on a regular grid andthe rate varies from around 1 point per second up to around 20,000points per second, depending on the technology being used, and theobject being scanned. The points from these scanning machines areaccurate to the order of 0.05 mm. CNC machines with probes scan byexecuting one or more programs that move the axes of the machine suchthat there is relative motion between the probe and the object.

CNC machines are expensive, partly because of the incorporation ofmotors and the associated equipment for assuring precision motion, suchas linear guides and drive screws. Few CNC machines are flexible enoughso that the probe can be oriented in six degrees of freedom so as toscan the complete surface of a complex object. Even when a CNC machinehas six degrees of freedom, it is often not sufficiently flexible so asto position the probe to scan the complete surface of the object withoutcolliding with the object. When the object is a person or expensive, therisk of using a CNC machine may be unacceptable, and there would be anecessity to make a machine to meet both the safety and scanningrequirements of the application. The programming of a CNC machine sothat the surface of the object is completely scanned without a collisionof the probe or machine with the object is often highly complex.Usually, the design of the machine and the degrees of freedom inherentin the design and limitations in the probe design, such as the standoffdistance during scanning between the probe and the object, mean that itis impossible to come up with a scanning strategy that will scan thecomplete surface of the object. It is common that the object has to bemanually picked up and replaced in a different position and/ororientation one or more times during scanning. Each time that thisoccurs, the object has to be reregistered to a uniform coordinate systemsuch that the data from the different scans can be accurately combined.

Manually operated scanning machines can be categorized into threetypes—horizontal arm machines, multiply-jointed arms, and devices basedon remote position-sensing means.

Manually driven, horizontal arm measuring machines usually have threeorthogonal axes and are usually based on a traveling column design.These machines are usually quite large, with the bed being at floorlevel so large items such as cars can easily be moved onto and off ofthem. Often motors can be engaged on one or more axes to aid the manualmovement of the machine. The probe is normally mounted at a fixedorientation on the end of the horizontal arm. This orientation may bechanged and various devices may be attached between the end of thehorizontal arm and the probe to aid the changing of the orientation,most of these devices having two axes. Horizontal arm machines have thedisadvantage of not being able to easily orient the probe in six degreesof freedom. The limited flexibility in the design of a horizontal armmachine makes most of the far side of the object unscannable.

Multiply jointed arms commonly comprise multiple linkages and areavailable for scanning complex objects. A multiply jointed arm typicallyhas six joint axes, but may have more or less joint axes. At the end ofthe multiply jointed arm, there is usually a tip reference point—such asa sphere whose center is the reference point, or a cone ending in apoint.

Scanning is carried out by bringing the point or sphere into contactwith the object being scanned. The computer monitoring the multiplyjointed arm then measures the angles at all the joints of the multiplyjointed arm and calculates the position of that reference point inspace. The direction of the last link in the multiply jointed arm isalso calculated. Positions can typically be output continuously at arate of around 100 points per second, but the rate can be much more ormuch less. The accuracy is of the order of 0.1 to 0.5 mm. The pointsfrom the arm are usually sparse and unorganized. The sparseness and lackof organization of the points make it difficult to provide enoughinformation for constructing a computer model of the object that is ofacceptable quality. A multiply jointed arm with multiple linkages has alimited working volume. In general, if a larger working volume isrequired, the arms become very expensive, less accurate and tiring, anddifficult to operate. The limited working volume can be increased byleapfrogging, in which the whole arm/base is moved to access anothervolume; but this requires a time-consuming system of registering atleast three points each time the arm is moved and recombining the datasets from each arm position. Manufacturers of multiply jointed armsprovide precalibrated arms and test methods that the user may employ tomake sure that the arm is still calibrated to an acceptable accuracy.Such test methods use, for example, the standard tip reference point atthe end of the arm and a reference sphere or a ball-bar, which is a rodwith two cylindrical cups that has a precise known distance between ahome ball and a end of arm ball. As the arm tip at the end of the ballbar is moved on the surface of a spherical domain, the arm recordspositions that are later compared to a perfect sphere, and errorestimates for the arm are output.

Remote position-sensing devices include hand-held devices that transmitor receive position information in a calibrated reference volume usingdifferent physical methods, including electromagnetic pulses and soundwaves. A hand-held device may be connected to the rest of the system bymeans of a cable. These devices are prone to generating scanned pointswith very large errors. Some devices cannot work when the object beingscanned has metallic components. They are less accurate than multiplyjointed arms, with accuracies of the order of 0.5 mm upwards.

There are three broad categories of scanning probes that could bemounted on the end of a multiply jointed scanning machine—point, stripe,and area probes. Point probes measure a single point at a time andtechnologies include mechanical contact methods and optical distancemeasurement methods. Stripe probes measure a number of points in a line,either simultaneously or rapidly in a scanned sequence. The most commonstripe technology is laser stripe triangulation. Area probes measure atwo-dimensional array of points on a surface, either simultaneously orin a scanned sequence. The most common technologies are interferencefringe and multiple stripe projection. Some area methods require thedevice to be held still for a few seconds during data capture. Stripeand area methods have an in-built speed advantage over point methods, asthere is less motion of the probe relative to the object. There aredifferences between the methods in terms of accuracy and cost, but thesedo not generalize with category. For example, a particular areatechnology may be cheaper and more accurate than another pointtechnology.

Means of capturing independent reference/feature points by contact arewell known and efficient. Structured light using stripe or area methodsis not good at capturing independent feature points because there is noway for the operator to align a known point on the object with a pointon the stripe or in the area.

Geometrical errors in the scanning process stem from many sources. CCDcameras can typically capture video at 25 frames per second. One majordisadvantage in normal use is that from any given demand for a frame,there is a variability of 40 msecs until the start of capture of thatframe. If the probe is being moved at, for example, 100 mm/sec, this canlead to a geometrical error of 4 mm in the probe's data. The duration offrame capture depends upon the shutter speed; e.g., 1/100 sec is 10msecs. One major disadvantage in normal use is that if the probe isbeing moved with a slow shutter speed, an additional geometrical erroris created. An arm is typically connected to a computer by a serialcable with arms typically generating positions at 125 positions persecond as they move. At this rate, there is a variability of 8 msecsbetween when a position is needed at the computer and when it arrives.This can also introduce a geometrical error when the probe is moving.The total variabilities of the CCD camera and the arm can cause largeaggregate errors.

There is a wide range of formats for 3D information in current use.These include the general categories—point formats, polygon formats, andcomplex surface formats.

Point formats include independent points, lines of points where a planeintersects with a surface, 2.5D areas of points that are commonly knownas range images that are single valued in Z and 3D point arrays, whichare often generated by medical scanners. The point formats have manystandard representations including the Range Image Standard (RIS)resulting from the European ESPRIT Research & Development Project 6911,IGES, and DXF published by AutoDesk, Inc., in the USA.

Polygon formats include polygons of different geometrical forms.Polygons may be 3- or more sided and formats may include mixed numbersof sides or always the same number of sides. Special cases, such asDelaunay triangulation, can specify the positioning of vertices and therelative lengths of the sides of polygons. Standard representations ofpolygon formats include STL, published by 3D Systems, Inc., in the USA;IGES; OBJ, published by Wavefront, Inc., in the USA; and DXF.

Complex surface formats include Bezier, NURBS, and COONS patches.Standard representations of complex surface formats include IGES,VDA-FS, SET, STEP, and DXF.

The objective of scanning can be simply to gather a number ofthree-dimensional points on the surface of the object, or it may be tocreate a computer model in a format that is useful for the applicationin which the model is to be used. It is generally true that a cloud ofpoints alone is not much use in many applications and that morestructure is needed to make a computer model efficient to manipulate intypical applications such as visualization, animation, morphing andsurface or solid modeling.

There are often benefits to be gained from reducing the size of thefiles associated with the model formats. Any file, whatever its formatcan be compressed using standard reversible utilities, such asPKZIP/PKUNZIP from PKWare in the USA. With 3D point arrays, an octetformat can be used to reduce the size of the arrays that represent asurface. An octet format splits a cubic volume into eight, smallercubes, and only further subdivides cubes by eight if they containinformation. An octet format is reversible. Moving from unstructuredpoint representation to polygon or complex surface formats oftenproduces large compressions but relies on approximations, so the processis nearly always irreversible. It is also difficult to automate so as togive good enough results. Chordal tolerancing is a commonly used methodof reducing the quantity of discrete points in a 2D or 3D polyline. Asan intermediate data structure, it has disadvantages in that theintermediate data structure does not record the orientation of eachstripe; it does not record breaks in the data, but assumes that all thepoints are connected by a surface; and it does not record jumps in thedata, such as those caused by occlusions.

Most scans today are carried out using a multiply jointed arm with a tipreference point. It is usual to first mark the object to be scanned witha permanent or temporary marking device, such as an ink pen or scribe,to create a polygonal network of splines. A single point is then scannedat each network intersection. On the computer, the points are linkedtogether into a polygonal structure. The overall process (marking,scanning, and linking) of creating a 3D polygonal model is at a typicalrate of 1 point (or vertex on the model) every 3 seconds. In someimplementations, the network is not marked on, but appears on a computerdisplay as each point is scanned. With this implementation, the networkis built up interactively. This method is suitable for models with arelatively small number of vertices, i.e., hundreds and thousands. Themethod is very slow, requires skill, patience, and concentration, and isexpensive in human time—particularly for large, detailed objects thatcan take three weeks to scan.

An alternative method of scanning with a multiply jointed arm andcontact tip reference point has often been tried, in which independentpoints are rapidly captured without the aid of a network. The points arethen input into a surfacing software package, which then constructs apolygonal network between the points. However, the “polygonization” ofunorganized data points is usually very slow, and speed decreasessignificantly as the number of points increases. The results are usuallyso poor as to be unacceptable. There is usually a significant amount ofhand editing of the data required.

Where a CNC scanning machine is used, the intermediate data structuresare usually range images. A number of unregistered range images may beregistered, polygonized, and integrated together. The raw data is anumber of range images of an object—typically from 5 to 20 innumber—with each one either being a cylindrical or a linear range image.The process is not automatic and requires a combination of operatorguidance and automated execution of algorithms. The operator first triesto align (i.e., register) the range images to each other on the computerusing a graphics display. This process is not accurate and is followedby an automatic least squares fitting process that attempts to adjustthe position and orientation of each range image such that they fittogether as well as possible. This process is lengthy, often takinghours on a powerful computer. Each range image is then independentlypolygonized into a network of 2.5D triangular polygons. Finally, thenetworks of triangular polygons are integrated. The output is a single3D polygon data set. The process is expensive, both in terms of capitalequipment cost and people time. It can take up to two years to becomeskilled enough to scan objects to produce good enough models. It canwork and produce good results for detailed objects.

For smooth objects, where the objective is to create complex surfaceformats, a coordinate measuring machine with a contact tip referencepoint is commonly used. It is usual to mark up the object with thedesired surface patch boundaries by using a marking device, such as apen or a scribe. These patch boundaries are then hand digitized with thecontact point probe. The software package then generates a CNC scanningprogram that automatically takes more points along the boundaries andinside the patches. The software then automatically generates a firstattempt at the surface model. This method is used because it is quickerand easier for the operator to define patch boundaries that will lead toa surface model with the desired structure before scanning, than todefine the patch boundaries after scanning using a software package on acomputer with a display showing the scanned points. It can take severaldays, and often weeks, to create patch boundaries that are usuallysplines, then create the patches and then trim the patches to form asurface model by using only the scanned points and a computer.

Scanned data points have been displayed in real-time. The display ofpoints has the disadvantage of easily becoming confusing to interpret,and also that the observer does not know when parts of the object'ssurface have been missed during scanning.

SUMMARY OF THE INVENTION

According to the present invention, there is provided a scanningapparatus for scanning an object to provide a computer model thereof,comprising means for scanning the object to capture data from aplurality of points on the surface of the object, where the scanningmeans captures data from two or more points simultaneously; means forgenerating intermediate data structures therefrom; means for combiningthe intermediate data structures to provide the model; means fordisplay; and means for manually operating the scanning apparatus.

The apparatus is most efficient in the quest to reduce the time and costof generating a computer model from a real-world object by means ofscanning with both time and cost reductions of an order of magnitudeachieved over conventional techniques. The model is generatedautomatically from the intermediate data in a form that may beimmediately usable in a wide range of applications.

The scanning means may use structured light to more quickly scan thesurface of the object. The scanning means may also be operable to sensethe color of the surface of the object, resulting in a model more likethe real-world object.

Preferably, the scanning means therein comprises means for generating asignal for scanning the object, signal detection means for detecting thesignal reflected from the object, and means operable in response to thedetected signal to provide the data for the intermediate data structure.

The structured light is preferably projected as a plane of light suchthat a stripe is formed on a viewing plane that is situated normal tothe projection axis of the signal generating means and situated at theaverage standoff distance from the signal generating means.

Alternatively, the structured light may be projected such that a patternis formed on an area of a viewing plane that is situated normal to theprojection axis of the signal generating means and situated at theaverage standoff distance from the signal generating means.

The signal generating means may be an illumination source such as alaser diode or one or more bulbs.

During scanning, the operator may see the surface he has scannedappearing in real-time as rendered polygons on the display such that hemay more easily scan the object. The operator may mount the object on aturntable, and then he may scan from a seated position rather thanwalking around the object. The scanning means can be mounted on manydifferent types of manual machines, giving enhanced flexibility forobjects ranging in size from small to very large. The scanning means canbe mounted on a multiply jointed arm for accurate scanning. The scanningmeans may be a self-contained unit that contains a remote positionsensor and incorporates a display to give the most flexibility inscanning. According to the invention, there is also provided a methodfor scanning an object to provide a computer model thereof, comprisingthe following steps:

-   -   Manually scanning the object with a signal by manual operation        of a signal generating means;    -   Detecting the reflected signal;    -   Generating intermediate data structures for the points;    -   Combining the intermediate data structures to provide the model;        and    -   Displaying the data, wherein the data is captured from a        plurality of points on the surface of the object simultaneously.

According to a further aspect of this method of the invention, the colordata is also captured from the object and then mapped on to the model.

Preferably, the data is displayed simultaneously as a plurality ofdisplay polygons.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing aspects and many of the attendant advantages of thisinvention will become more readily appreciated as the same become betterunderstood by reference to the following detailed description, whentaken in conjunction with the accompanying drawings, wherein:

FIG. 1 is a schematic representation of a scanning apparatus accordingto the invention;

FIG. 2 is a schematic perspective drawing of a probe;

FIG. 3(a) illustrates a first embodiment of the configuration of theoptical elements housed in a probe;

FIG. 3(b) illustrates a lamp configuration;

FIG. 3(c) illustrates an alternative to the lamp configuration of FIG.3(b);

FIG. 3(d) is a graph illustrating intensity as a function of distancealong the line A1-A2 of FIG. 3(a);

FIG. 4 illustrates a second embodiment of the configuration of theoptical elements housed in a probe;

FIGS. 5(a) to 5(d) illustrate a method of calibrating the color of thescanning apparatus of FIG. 1;

FIG. 6 is a schematic block diagram illustrating the capture of colorand position data;

FIG. 7 is a schematic representation illustrating how the detection ofthe color and position data is synchronized;

FIG. 8 is a schematic representation of the end of the multiply jointedarm of the apparatus of FIG. 1;

FIG. 9 is a schematic illustration of the turntable and multiply-jointedarm of the apparatus of FIG. 1;

FIG. 10 illustrates the mounting of the probe on the multiply jointedarm;

FIG. 11 illustrates the alignment of the mount on the multiply jointedarm;

FIG. 12 illustrates the alignment of the probe on the multiply jointedarm;

FIG. 13 illustrates a linear range image;

FIGS. 14(a)-14(b) illustrate cylindrical range images;

FIG. 15 illustrates the range image placing method;

FIG. 16 illustrates the surface normal extension method;

FIG. 17 represents the structure of a single point in a range image;

FIG. 18 illustrates the representation of an object by three rangeimages;

FIG. 19 illustrates the range image updating method;

FIGS. 20(a)-20(b) illustrate first and second stripes captured on a CCDarray;

FIGS. 20(c)-20(f) illustrate the respective captured data points andstrings of data points from the first and second stripes of FIGS. 20(a)and 20(b);

FIG. 20(g) illustrates polygons generated from these strings;

FIG. 21 illustrates a probe mounted on a head and a heads-up display;

FIG. 22 illustrates color image mapping;

FIG. 23 illustrates the timing for position interpolation;

FIG. 24 illustrates the triggering of the arm position measurement;

FIG. 25 illustrates an object with marked lines thereon;

FIG. 26(a) illustrates a probe mounted on a multiply jointed arm, whichis mounted on a horizontal arm machine;

FIG. 26(b) illustrates two opposing horizontal arm machines;

FIG. 27 illustrates a human foot being scanned;

FIG. 28(a) illustrates stripe sections of a pipe network and panel;

FIG. 28(b) illustrates partial polygon models of a pipe network andpanel;

FIG. 28(c) illustrates extrapolated polygon model of a pipe network;

FIG. 29(a) illustrates stripe scanning; and

FIG. 29(b) illustrates area scanning.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Referring now to FIG. 1, a scanning apparatus 100 comprises amultiply-jointed arm 1 with an arm control unit 2 and a probe 3. Thecontrol unit 2, which includes a processing unit 10, is coupled to acomputer or processing unit 4 and color monitor 7. The probe 3 is alsocoupled to a probe control unit 5 that is likewise coupled to thecomputer 4. The intermediate data is displayed on the color monitor 7 asrendered polygons 13. The probe 3 provides a stripe 8, which isprojected onto an object 9 positioned on a turntable 14. The stripe 8 isin the form of a plane of light. Buttons 6 are also provided to controldata capture. A color frame grabber 11 in the computer 4 is mounted on astandard bus 12 and coupled to the probe 3.

The computer 4, probe control unit 5, arm control unit 2, buttons 6,color frame grabber 11, and monitor 7 are provided separately. Forexample, the computer 4 and monitor 7 may be a personal computer andVDU, although for certain applications, it may be more convenient forone or all of them to be provided on the probe 3.

The multiply jointed arm 1 and the probe 3 are coupled to the computer 4by means of the control units 2, 5 discussed above. The computer 4receives information from the scanning stripe 8, theposition/orientation of the arm 1 in terms of X,Y,Z coordinates, withthe coordinates I,J,K of the surface normal of the probe 3 and colordata if required.

Referring now to FIG. 2, an embodiment of the probe 3 for use withremote position sensors 261 is shown. The probe 3 is lightweight andresilient so as to withstand being knocked without losing itscalibration.

Referring now to FIG. 29(a), the structured light is preferablyprojected as a plane of light 364 such that a stripe 8 is formed on aviewing plane 360 that is situated normal to the projection axis 361 ofthe signal generating means 362 and situated at the average standoffdistance S from the signal generating means.

Referring now to FIG. 29(b), the structured light may be projected suchthat a pattern 363 is formed on an area 365 of a viewing plane 360 thatis situated normal to the projection axis 361 of the signal generatingmeans 362, and situated at the average standoff distance S from thesignal generating means. The pattern 363 in this example is a number ofstripes that may be of different colors.

Two alternative embodiments of the probe 3 are described. Now referringto FIGS. 3(a)-3(d), one embodiment of the probe 3 is described. Theprobe 3 comprises a number of components mounted on a base plate 20. Astripe generator 22—for example, containing a laser diode—provides thestripe 8 for projection onto the object 9 to be scanned. Typically, thelaser will be of Class 2 or less, according to the CDRH 1040.11classification in the USA, viz. less than 1 mW in power at 670 nmwavelength. The stripe 8 is nominally focused at some point P. A lensassembly 24 is used to focus the image onto a high resolution CCD camera25. The camera 25 may be oriented at an angle, satisfying theScheimpflug condition. An optical interference notch filter 26 is usedto selectively image light of the wavelength of the stripe 8. A simpleglass cut-off filter 27 reduces ambient light within the probe.

Information on the color of the surface of the object may be recorded inintensity scales or color scales, such as RGB.

An intensity scale estimate of the color of the surface may be obtainedby recording the reflected light level of the stripe as it is imaged onthe high resolution CCD camera 25 at each point. A high level indicatesa light surface at that point that scatters much of the projected light,and a low level indicates a dark surface at that point that absorbs muchof the projected light. These indications may be false, such as onspecular surfaces, as known to a person skilled in the art.

A color estimate of the color of the surface can be obtained by means ofa color camera 29 comprising a color CCD array. Color scanning requiresthat the object be lit. Lighting may be by means of ambient light fromexternal light sources or by lamps situated on the probe. There areseveral disadvantages in using ambient light only for color scanning.First, ambient light intensity varies over nearly every object in astandard environment such as a room with overhead lighting. Second, itcan be a time-consuming procedure to position a number of lights so asto evenly light the object. Third, the probe itself can cast shadowsonto the object.

Four lamps 28(a)-28(d) are provided around the lens 31 of the camera 29for illumination, or a ring lamp 28′ could be used. This configurationis used to avoid any problems of shadowing. The lamps may includerespective back reflectors 32a-32d, where appropriate. The lamps are setto give an average intensity of around 80-150 Lux, but the intensitycould be much more or less and, during use, ambient light is reducedsignificantly below this level—for example, by dimming or switching offoverhead lights. This removes any effects from variations in ambientlight. The lamps may be tilted with respect to the camera axis to ensurethat light of a more even intensity is projected onto the object 9 atthe average scanning standoff distance. The lamps should be small insize to obtain the least possible weight penalty, especially if two ormore lamps are used. To extend their life, they can be operated at alower voltage at certain time periods—for example, while preparing forthe capture of each image. When the operator triggers a color imagecapture, the voltage to the lamps can be momentarily increased tomaximum. The lamps are only switched on for color capture. During theprocess of 3D capture, the lamps are switched off where this will alsohave the added advantage of increasing the signal to noise ratio. Anaccess panel 35 can be provided over the lamps so that the lamps can beeasily replaced without opening the probe and risking losing itscalibration. To improve results when scanning a reflective surface,polarizing material 34 is placed between the camera 29, lamps 28a-28d,and the object 9. To reduce variations in the projected light, diffusers33a-33d are placed in the light path between each lamp and object 9 or,alternatively, the lamp glass or back reflectors are treatedaccordingly.

Referring now to FIG. 4, the second embodiment of the probe 3 isdescribed. As also preferable with the first embodiment, the base plateis provided with a removable two-piece cover 21 mounted on the baseplate 20 to define a housing for the components of the probe 3 toexclude ambient light and to protect the components. It may have ametallic interior coating to reduce electromagnetic emissions andsusceptibility. The cover is appropriate for both embodiments.

The probe 3 in the second embodiment can also capture the color of thesurface of the object 9. The color is captured using an optical systemthat is coplanar with the light stripe 8. The main advantage ofcoplanarity is that it provides the color of each point directly,whereas non-coplanar systems, such as in the first embodiment, requireextensive post-processing computation to map the captured color dataonto the captured 3D data. In non-coplanar systems, the whereabouts ofthe stripe in the color camera is significantly variable due to thenon-alignment of the two cameras, leading to an embodiment that isoperated for scanning in two passes—3D and color—instead of the one passthat is achievable from the first embodiment.

During use, with the stripe generator 22 switched off and the lamp 28switched on, a color sample 32 of the object 9 is directed back alongthe direction of where the stripe 8 would be if the stripe generatorwere illuminated where it is reflected by a half-silvered mirror 30 andfocused onto the color camera 29 via a lens assembly 31.

In a probe 3, where the stripe generator 22 produces a white stripe, thecolor and position can be captured synchronously. The environment wouldneed to be dark in this case, and lighting 28 would not be required.

A look-up method is provided for determining where to read the colorfrom the rectangular array of the color camera 29, depending on theobject distance S at all points along the stripe 8. Now referring toFIGS. 5(a)-(d), a color camera look-up table 166 is prepared by viewingthe stripe—when it is permanently illuminated—at a sufficient number ofpoints in a rectangular array covering the distance measuring depth andthe stripe measuring length. A fixed-length, flat object 161 is one itemthat can be used for this purpose, and it typically has a white surface.The flat object 161 is placed on a light stripe absorbent background162. The probe 3 and the flat object 161 move relative to each other inthe direction of the W axis such that stripes 163 are imaged. The colorcamera image 165 shows the imaged stripes 163a collected in a region 164of the array. In the case of perfect coplanarity, the imaged stripeswill be superimposed (see 163b in FIG. 5(c)). The look-up table 166 isthen built up so that, for a scanned point 167 on an object withcoordinates V1,W1, will have a color image position Cx,Cy in the look-uptable 166 that stores an array of values of Cx,Cy for the V,W ranges.During scanning, the color image is usually scanned before the stripemeasurement. The extent of the look-up table 166 can determine how muchof the color image 165 needs to be stored while the points are beingcalculated, i.e., the extent of region 164. This reduces the amount ofmemory needed in the computer 4 and the required bandwidth fortransferring color information from the camera 29 into the computer 4,allowing the possible use of lower cost units, such as the frame grabber11 located in the probe control unit 5, or in the computer 4 on a bus,as discussed above. It is probably not worth the expense of building aperfectly coplanar system, as a roughly coplanar system can becalibrated as described above to produce as effective a result.

Referring now to FIGS. 6 and 7, on coplanar systems, if the light stripe8 is of a color other than white, then it is difficult to capture theobject's surface position and color at the same time. To overcome thisproblem, the light stripe 8 and color camera 29 can be switched on andoff such that the color is recorded shortly before or shortly after theposition. Adjusting circuitry can be provided to change the exposuretimes of the color camera 29 and the stripe generator 22 and to ensurethat the gap between the color capture and the stripe capture isminimized. To achieve this, a video synchronization generator 60generates a synchronization signal 61 of pulses at video camerarate—which, using the CCIR format, is 50 times per second. Thesynchronization signal 61 is fed into the high-resolution camera 25 andthe color camera 29. The exposure time of the color camera 29 can be setmanually using a switch 63 or remotely with an electrical signal. Thesynchronization signal 61 is also fed into a standard circuit 62, whichswitches on the stripe generator 22 for a period of time after aninitial delay. The period of time that the stripe generator is on may beset manually using a control 64, and the delay between when thesynchronization signal 61 is received and the stripe generator 22 isswitched on may also be set manually using a control 65. With referenceto FIG. 7, the synchronization signal 61 is represented by the traceSYN. The illumination of the stripe generator 22 is represented by thetrace L. The exposure of the color camera 29 is represented by the traceC. The exposure of the high-resolution camera 25 is represented by thetrace M. This is only one way by example of controlling coplanar probes.

Referring now to FIG. 8, the probe 3 projects the stripe 8 so that themeasuring area starts before a tip reference point 51 provided at theend of the multiply-jointed arm 1 and extends a further distance. Thetip reference point 51 may be the tip of a cone, a sphere, or a rollingdevice, such as a rolling wheel or ball, or the point may be anythingsimilar providing a tip. In the embodiment described herein, the tipreference point 51 is the center of a sphere 52. This tip referencepoint 51 enables the operator to scan hard objects by tracing the sphere52 along the object 9 in strips in contact with the object 9. For softobjects, the sphere 52 acts as a scanning guide, and typicalinstructions might be to keep the tip reference point 51 about 20 mmfrom the object 9 while scanning. In this way, the probe 3 may be keptclose enough to the object 9 for the stripe 8 to be in the measuringarea but without touching the soft object 9. The stripe 8 typicallystarts 100 mm from the tip reference point 51 at the end of the arm 1,but could be much closer or further away, and can be used to measureobjects lying between the two points W1 and W2, as measured from the end55 of the probe 3. The ideal method—from a usability point of view—isfor the plane of the stripe 8 to be coaxial with the axis 54 of the lastsection 50 of the arm 1. This is the case for a purpose designed arm anda hand-held probe. The probe 3 may often be retrofitted onto the arm 1,and because a mechanical arm has a diameter of typically 20-60 mm, thispresents an alignment problem. In this case, the plane of the stripe 8is not coaxial, but may be either in a plane 53 parallel to the arm endaxis 54 or in a plane 53a angled to this axis 54, so as to cross theaxis 54 at some point. By crossing the axis 54 of the arm 1 somewhere inthe measuring range, the ergonomics of the arm 1 can be enhanced becausethe light plane is in an easier to use position. This crossover istypically somewhere between the tip reference point 51 and the end W2 ofthe measuring range.

Referring now to FIG. 9, the use of a manually rotated turntable hasseveral advantages. For a given arm size, larger objects can be scanned.The operator does not have to move around the object. This makesscanning physically easier, more enjoyable, and there is less chance ofeither the operator or the arm accidentally knocking the object or ofany reference point being lost.

The position and coordinate system of the turntable 14 must be knownrelative to that of the arm 1. The tip reference point 51 can be placedin a locating cone or cup 202 in the table at a large radius. Points aretransmitted regularly by the arm control unit 2 and recorded on thecomputer 4 as the turntable 14 is manually rotated. Functions that fit aplane and a circle through these points provide complete position andorientation information on the turntable 14 in the arm coordinatesystem.

During scanning, it is important to know the turntable angle. Theturntable may be designed to have precise mechanical resting positions206a-206d, e.g., every 15 degrees. These resting positions 206 would beapparent from a pointer 208 indicating the angle on an attached scale209 of 360 degrees. The operator could type in the new angle into thecomputer each time the turntable was rotated. However, the process oftyping in an angle means that the operator may have to put down theprobe 3. This slows down scanning, and there is scope for an error to bemade by the operator.

With an electrical connection 204 between a position sensor 203 on theturntable 14 and the computer such that the computer could know eitherprecisely or roughly the turntable angle, the process is faster and lesserror prone. If the sensor 203 is accurate—such as an encoder with, forexample, 10,000 lines—then the turntable 14 could be positioned at anyorientation and its angle known precisely. This allows for scanningwhile rotating the turntable, although care must be taken that dynamicsdo not lead to position errors or to the object moving relative to theturntable. If the sensor 203 is less accurate—such as apotentiometer—then the turntable 14 could also have precise mechanicalresting positions 206. This gives the advantages of high accuracy andlower manufacturing cost. Each time the probe 3 captures data from theobject 9, the software must check for movement of the turntable 14. Ifit has been moved then with a less accurate turntable sensor 203, probedata should be thrown away until the turntable 14 has stopped moving. Inall cases, the turntable 14 should be capable of being operated by onehand such that the probe does not have to be laid down. It is often thecase that an object on a turntable is scanned with regular increments,e.g., eight scans every 45 degrees. To aid the operator in incrementingby X degrees, different shaped and/or colored icons could be placedevery X degrees on the scale and on the other regular intervals. Typicalintervals might be 45, 60, 90 degrees. With reference again to FIG. 2,this method can also be used with a probe 3 including one or more remoteposition sensors 261 with a tip reference point 51. The manual turntablemay be driven by a motor operable by means of hand controls.

Each time a probe 3 is mounted on the arm 1, if the mounting is notrepeatable to a high accuracy, then the transformation in six degrees offreedom between the arm coordinate system X,Y,Z and the probe coordinatesystem U,V,W will have to be found.

A mounting device 210, 214 for the probe is illustrated in FIG. 10.Accurate and repeatable geometric positioning of the probe on the arm isrequired. This is provided by the mounting device 210, 214. The mountingdevice 210, 214 provides a standard mechanical interface that maypreferably be used for all probes and arms, which is both small andlight, and that is easy to use to mount and dismount the probe onto andoff the arm. The mounting device comprises a arm side mount 210 thatcomprises a flat mating surface 211 with two precisely dimensionedprojections 212 located in precise positions on the mating surface 211.The mounting device also comprises a probe side mount 214 comprising aflat mating surface 215 and two precisely-dimensioned recesses or holes216 corresponding to the two projections 212 in the arm side mount 210.It is essential that the geometric repeatability in position andorientation is very high.

A standard mounting device between any arm and any probe gives severaladvantages. When the arm has to be used without the probe, then theprobe has to be removed. If the mounting device is not repeatable, thenthe system will require realignment before use each time the probe isremounted.

Typically, a range of probes will be supplied with different weights,speeds, sizes, and accuracies corresponding to different functions. Eachprobe can be supplied with alignment data relative to the datum of theprobe side mount 214 and, in this way, any probe may be attached to thearm without the requirement of realignment. A user may also have one ormore different arms. In order to fit the same probe to two differentarms, the user needs only to acquire an extra adapter for the secondarm, which would fit onto the arm and include the arm side mount 210 ofthe mounting device 210, 214. The arm side-mounting device 210 may beattached to any machine, including multiply-jointed arms, horizontalarms, and orientation devices such as manual, two-axis orientationdevices.

To calculate the six degrees of freedom transformation between the armcoordinate system and the probe coordinate system, one can either treatit as one transformation or as a multiplication of two transforms if theaccurately repeatable mount is considered as an intermediate referencepoint, i.e., the transformation matrix Tap between the arm and the probecoordinate systems is equal to the transformation matrix Tam between thearm and the mount coordinate systems multiplied by the transformationmatrix Tmp between the mount and the probe coordinate systems:Tap=(Tam)·(Tmp)

The transformation matrix Tam can be found in several ways. Nowreferring to FIG. 11, a particularly simple, cost effective, andpractical method involves the use of a reference plate 220. Thereference plate 220 has three orthogonal flat surfaces 222 and amounting point 221 to which the arm side mount 210 can be attached in aprecisely known position relative to the orthogonal planes. Tam can becalculated using the following steps:

-   -   The reference plate 220 is fixed so that it cannot move relative        to the arm coordinate system;    -   The arm side mount 210 is fixed rigidly onto the arm 1 (if it is        not already present), without the probe attached;    -   The three orthogonal planes 222 of the plate 220 are measured by        the tip reference point 51 on the arm such as to fully define        the position and orientation of the reference plate;    -   The arm mount is then mated with the mounting point 221 on the        reference plate 220;    -   The arm position and orientation are recorded; and    -   The transformation matrix Tam is then calculated from the known        geometry of the reference plate 220 and the measurements from        the previous steps.

The above method can be encapsulated in the main scanning softwareprovided with the scanning system or in a separate program. This has theadvantage that much time is saved over an alternative of the usercalculating Tam manually from arm positions output by the armmanufacturer's software and manually inputting the resulting Tam intothe main scanning system software.

The probe side mount 214 is integral to the probe and does not moverelative to the probe coordinate system. The transformation matrix Tmpis provided by the probe supplier with the calibration data for theprobe.

The direct calculation of Tap using the arm and probe coordinate systemsbut without involving an intermediate mount can be carried out in manyways. Most of the ways involve using the probe mounted on the arm tocapture data from one or more geometrical objects. The problem hasproven to be very difficult, since many of the standard methods produceinaccurate results in either the orientation or position componentsoften due to inherent instabilities triggered by relatively smallerrors. One way is disclosed, by example, for an S stripe probe:

-   -   Now referring to FIG. 12, the transformation matrix Tap is        calculated by:    -   1. Mounting the alignment calibration object with three        orthogonal faces 230 so that the three orthogonal flat surfaces        231, 232, 233 are reachable and accessible by the probe 3        mounted on an arm 1;    -   2. Capturing at least three stripes from three different        orientations on the first flat surface 231. The orientations        need to be different enough to provide stability to the        mathematical algorithm (in practice, a variation of at least        five degrees between every two of the stripes is sufficient);    -   3. Repeating this three or more stripes capture on a second flat        surface 232 and on a third flat surface 233; and    -   4. Processing the data from the nine or more stripes in an        iterative fashion to output Tap.

The handedness of the coordinate systems of the arm 1 and the probe 3would be known. The relationship between the normals of the surfaces onthe alignment calibration object 230 could be specified. One way ofdoing this is by labeling the three faces 231, 232, 233 and specifyingthe order in which the three faces must be scanned.

The main advantages of the above apparatus and its method of aligningthe probe are (1) that it involves a single alignment calibration objectthat is cheap to manufacture to the required geometrical tolerance andis relatively light and compact; (2) that the method is robust, simpleto carry out from written instructions and quick; (3) that theprocessing can be encapsulated in the main scanning software providedwith the scanning system or in a separate program; (4) that there is noneed to have any preliminary geometric information about the orientationand position of the probe relative to the tip of the arm at the start ofthis method—for example, the probe could be slung on the underside ofthe arm pointing backwards and the method would work; and (5) that ifthe probe is knocked or damaged such that Tmp changes but thecalibration is still valid, then this method of alignment will stillwork.

In using scanning systems to provide data for 3D applications software,the need for specific 3D reference points in addition to 3D surfacesbecame apparent. Some applications for which 3D surfaces are requiredthat also require 3D reference points are animations involving jointmovements where a joint is to be specified in the context of the 3Dmodel. In this case, the joint can be quickly defined from one or more3D reference points. A new method of using the scanning system is to usethe probe 3 to scan the surface and to use the tip reference point 51 tocapture individual 3D points by contact. An alternative method is toproject a calibrated crosshair onto the object and use an optical methodof picking up individual points. This can be used in both stripe andarea systems. The calibrated crosshair is usually switched on justduring the period in which individual points are captured. There couldbe two modes—in the first mode individual points are captured each timea button is clicked; and in the second mode, a stream of individualpoints are captured from when a button is first pressed until it ispressed again. The second mode is commonly used for tracing outimportant feature lines, such as style lines or patch boundaries. In thecase of a stripe sensor, instead of projecting a crosshair, it may onlybe necessary to project a second stripe at the same time as the mainstripe. The crosshairs may be calibrated by the probe supplier using athree-axis computer controlled machine, a known calibration object, andstandard image processing techniques.

The scanning apparatus 100 is operable to scan an object and therebygenerate a computer model of the object's surface using an intermediatedata structure for efficiently storing points on the surface of theobject during scanning, creating an instance of the intermediate datastructure for the particular object; and controlling the storage of thescanned points in the intermediate data structures during scanning withan operator control system.

Three examples of these intermediate data structures may be points orencoded stripes or range images.

Points have the disadvantage of being unorganized and much informationobtained from the structure of the probe and the method of its use islost if the 3D data is reduced to points.

In the case of stripe probes, much information may be retained toimprove the speed and quality of construction of a model fromintermediate data if an encoded stripe intermediate data structure isused. Such a structure stores data from one stripe at a time. Thestripes are stored in the order of capture. The time of capture of eachstripe is recorded. The orientation of the probe is recorded for eachstripe. The raw data points from the stripe may be processed beforestoring in the data structure to determine jump and break flags and tosample or chordally tolerance the raw data points to reduce the size ofthe intermediate data structure without losing any significantinformation.

In the case of area probes, the advantages of a range image as anintermediate data structure are well known. These advantages include adata structure that relates well to the area based data capture methodand the efficiency of storage in an image in which only Z values arestored.

An intermediate data structure can be used in which the surface of anobject is described by means of a finite number of linear andcylindrical range images that are, to some extent, characterized by theshape of the object.

A linear range image 70 is illustrated with reference to FIG. 13. Therange image 70 has a coordinate system U,V,W and a spacing of points dUin the U direction and dV in the V direction. The linear range image 70contains in its header definition its relationship to the worldcoordinate system X,Y,Z, i.e., the arm coordinate system. In thedisclosed invention, the linear range image 70 cannot store negative Wvalues.

Cylindrical range images 71, 72 are described in FIGS. 14(a)-14(b). Therange image has a coordinate system W,R,A where A is an angle. Thespacing of points is dW in the W direction and dA in the A orientation.The cylindrical range images 71, 72 contain in their header definitions,their relationships to the world coordinate system X,Y,Z. In thedisclosed invention, the cylindrical range images 71, 72 cannot storenegative R values. The direction +R and position R=0 of a cylindricalrange image defines whether the points stored are inside the rangeimage, as in FIG. 14(a), or outside, as in FIG. 14(b).

Referring now to FIG. 15, the range image placing algorithm takes ascanned point and tries to place it into defined range images byprojecting a ray along the normal to the range image 105. If the pointhas a negative value in a range image—for example, point 104—then it isnot stored in that range image. If the point is outside the extent ofthat range image, it is not stored in the range image unless the rangeimage is extensible—in which case, the algorithm extends the range imagefar enough to place the point. If a point already exists in the rangeimage position in which the point is to be placed, then the two pointsare compared. If the distance between the two points in space is outsidea tolerance d, such as the error of the scanner, then the nearest point102 is stored and the furthest point 101 is rejected. If the two pointsare within the error of the scanner, then their values are averaged andthe average value is stored.

The range image-placing algorithm is simple and quick, but it isindiscriminate, often placing points incorrectly in range images andrelying upon them being overwritten by a nearer point. If the rangeimage is very dense, but populated with few values, then up to half thepoints populated could be incorrect because the surface normal of thepoint is incorrect. This can restrict successful scanning to coarserange images.

The range image-placing algorithm is improved upon with the surfacenormal extension. The range image-placing algorithm does not have anestimate of the surface normal of the point to be placed. Also, it doesnot take into account the orientation of the probe when the stripe iscaptured. To improve the range image placing, the fact that most stripesare scanned in sequence and have near predecessor and near successorstripes is used. For example, as illustrated in FIG. 16, there are eightneighboring points 116 on a stripe 114 and on its predecessor 113 andsuccessor 115. These can be used to approximate the surface normal of apoint P before it is placed in a range image. Three sequentially scannedstripes 113, 114, 115 are shown on an object 111 and projected onto arange image 112 as stripes 113a, 114a, and 115a. The point P, withcoordinates Xp,Yp,Zp on stripe 114, has eight near neighbors 116 on therespective stripes 113, 114, 115 as described above, and an approximatesurface normal Np with coordinates Ip,Jp,Kp. The probe orientation forstripe 114 is N_(S) with coordinates Is,Js,Ks. By calculating thesurface normals N_(S), N_(P), and N_(R), where N_(R) is the normal ofthe range image 112, one is given a choice of two opposite surfacenormals. The correct one is the one that can be seen from the probe 3orientation—assuming that the changes in probe orientation for the threestripes are not significant to the surface normal direction. If thesurface normal N_(P) of a point P is found to be facing away from thesurface normal N_(R), then the point is not placed on the range image.This surface normal extension eliminates the majority of incorrect pointplacements in range images. In a practical implementation, three stripesof points are buffered before the first stripe of points is placed inthe range images. The normal extension in a modified form can also beused for the first and last stripes by using the two successive or twoprevious stripes. When the three stripes 113, 114, 115 are nearlycoincident, perhaps because the arm is moving too slowly, then theaccuracy of the surface normal estimate is low and the normal cannot beused. A different normal calculation can be made using any neighboringpoints already placed in the range image instead of the neighboringstripes. A further, normal extension to the range image placingalgorithm combines both the stripe and the range image data to provide abetter estimate of the surface normal. The calculations involved inthese normal extensions can provide a bottleneck to the scanningprocess. The bottleneck can be overcome by using only two stripes, lesssamples (5 instead of 9) or a faster computer.

A number of range images that are positioned in the object coordinatesystem must be defined. The range images have specific mathematicaldefinitions. Two basic types of range image are used—linear andcylindrical—as discussed above. A range image has direction and a zeroposition. The range image can only store points that are in front of itszero position. If there are two or more surfaces of the object in linewith a point in the range image, then the surface that is nearest to therange image's zero position is represented in the range image. A rangeimage can be constrained in size or unconstrained in size. The rangeimage can be one image of fixed density or comprise a patchwork of anumber of adjoining images of different densities. Each grid position inthe range image is single-valued. The range image will typically use 4bytes to store a depth value Z, from 1 to 4 bytes to store the grayscale or color value 1, and from 1 to 3 bytes to store the orientationN. This is illustrated with reference to FIG. 17, which illustrates howa single point is represented. The 3 bytes suggested for orientation Nwill not permit a very accurate orientation to be stored. More bytescould be used, but there is a trade-off between data storage size,processing time for converting floating number orientations to/from acompressed integer format, and accuracy. Range images will normallyrequire from 5 to 11 bytes to store each point, depending on theoperator's requirements. For comparison, 20 bytes are typically requiredto store an ASCII X,Y,Z value.

Now referring to FIG. 18, it is possible to define—for a finite object 9of any shape—a finite number of range images of the above types that,for all practical purposes, enables any and every point on the externalsurface of the object 8 to be stored in one or more range images 81, 82,83.

With objects with deep external features, such as the inside of an ear,it may not be possible or practical to scan all parts of the externalsurface, but it is possible to represent them theoretically.

The number and position of the range images used in the process are suchthat they are sufficient to be able to store enough of the surface ofthe object to enable a computer model of the desired accuracy and detailto be generated.

In a manual process, the number and position of all the range images maybe defined by the operator before scanning. Alternatively, just one maybe defined by the operator before scanning begins, followed by thedefinition of others at any point during scanning. The operator has achoice of several strategies. He can define range images and scan rangeone at a time. He can define a number of range images and scansimultaneously. He can define some range images and scan followed bydefining more range images and then scanning. If a point is scanned thatdoes not fit onto any defined range image, then it is rejected.Alternatively, such rejected points could be automatically saved forplacing into any new range images that the operator may subsequentlydefine.

A typical number of range images varies from 1 to 20. Some range imagesneed only be very small in size—small enough to cover a part of theobject that is otherwise hidden from recording on other range images.The density of each range image can vary. For instance, a large, smoothpart of the object does not need a high point density; but a small,finely detailed ornament may require a high point density. Each rangeimage has a direction.

The operator may select the most suitable set of predefined range imagesfrom a library of range image sets. He can then edit the set to suit hisobject. Each new set is then stored in the library. A set can be thoughtof as a set of templates. As an example, for a human form there could bea range image set consisting of five cylindrical range images for thelimbs and the trunk, together with five linear range images for the topof the head/shoulders, hands, and feet. For a car, one cylindrical rangeimage for the car's body and two linear range images at each end of thecar could be enough. It is important to note that the axis of acylindrical range image must lie within the object or part of the objectbeing scanned.

A range image is manually defined by the operator by first selecting theappropriate range image type—cylindrical or linear—and second, placingthe probe to give the desired position and orientation of the rangeimage and selecting it using the operator control system. For acylindrical range image, the probe could be positioned to first give theposition and direction of the axis and then to give the maximum radius.

Now referring to FIG. 19, an inference method provided for updatingrange images from the other registered range images is also a novelmethod. The inference method progresses through each array position inthe range image 121 that is to be updated. The inference algorithm canupdate positions that either have no value or have a value with asurface normal that is steeper than a given value, or have a less steepvalue, or any combination of these according to the operator'srequirements. If a position in the range image 121 is to be updated,then that position is projected as a normal ray 126 onto all the otherrange images 120, 125, one at a time. If the ray intersects with anotherrange image 120, then the local triangular surface element through whichthe ray first passes is located on the surface 123 and constructed. Thevalue 124 at the intersection of the ray 126 and the triangular element122 is then inferred and placed onto the range image being updated. Ifthe ray intersects several range images 120, 125, then the inferredvalues from the range images are averaged after outliers have beenremoved. Outliers are removed by using a tolerance such as the error ofthe scanner. The original value (if it exists) in the range image 121being updated, could be included in this outlier removal/averagingprocess.

The inference method is particularly used when an additional range imageis added at a late stage in the scanning process or if range images aredefined/scanned one at a time. The method enables surface areas that arenearly orthogonal to the range image, i.e., are almost vertical walls,to be well defined from data stored in the other range images. Thisprovides a better set of points for carrying out the polygonization ofone range image resulting in a more accurate polygonal network andsimplifying the polygonization process.

The probe 3 provides data that is displayed on the display monitor 7 asa rendered polygonal surface 13 in real-time or with an acceptable delaysuch that the user can watch the display monitor 7 and use the feedbackof the rendered surface to guide his movement of the probe 3. Real-timeis defined in the context of visualization as an operation reacting witha delay small enough to be acceptable to an operator in normal use. Theprobe 3 could be a stripe probe or an area probe. Where the probecaptures 3D and color information, then the color information can bemapped onto the 3D model to texture it, as discussed below.

The surface to be displayed is calculated for stripe probes oneadditional stripe at a time. Referring now to FIGS. 20(a)-20(g), as astripe 301 is captured, it is converted using one of several commonlyused methods into a finite string 303 of 3D points 302a, 302b and flags304, 305 in the world coordinate system X,Y,Z using the previouslyobtained calibration and alignment data for the probe 3. The maximumnumber of points is usually equal to the number of rows 281 or thenumber of columns 282 in the CCD array 25, depending on the orientationof the CCD in the optical setup. This disclosure will refer to rows, butit can equally apply to columns or any other way of organizing the datarecorded by the camera. Where the stripe crosses a row the position onthe CCD array can usually be calculated to subpixel accuracy by one ofseveral commonly used methods. If there is no data for one or moreneighboring rows, such as missing positions 302e, 302f (not shown), thena “break” flag 304 is output into the string to indicate a break in thesurface recorded. If there is a significant jump discontinuity in rangeabove a maximum value that is appropriately set for the scanningresolution of the probe 3, such as between 302j and 302k, then a “jump”flag 205 is output into tile string of 3D points to indicate either avertical wall relative to the probe orientation or an occluded surface.The string 303 is filtered to reduce the number of points whileeffectively transmitting most of the information. The object offiltering is to reduce the amount of data processed for surfacerendering and hence increase the speed of the rendering process with aminimal degradation in the quality of the surface. The first method offiltering is to skip some of the stripes. Another method is to samplethe rows, e.g., take every nth row. A third method is to chordaltolerance all the points in the stripe and discard the points that aresurplus and within tolerance. Where computer speed is limited, the firstand second filtering methods are preferred because of their simplicityand because the resulting regular grid of points produces regularpolygons that look good on the display, as opposed to long thin polygonsthat might result from a chordal tolerancing process that can haverapidly changing surface normals if the data points are slightly noisydue to inaccuracies in the probe and the arm, and may present anunattractive “orange peel” effect on the display. The same process isrepeated for a second stripe 306 capturing data points 307, resulting ina second string 308 of 3D values 307a, 307b, etc., and flags. A surfacecomprising triangular or quad polygons is then constructed between thetwo strings 303 and 308, resulting in a string of polygons 309. Thestring of polygons is then displayed by a renderer. The renderer may ormay not take into account the previous polygons displayed, theviewpoint, and lighting model.

If color has been recorded for a polygon then the color information canbe mapped onto the polygon. The precise mapping algorithm depends on theformat of the raw color information, which depends on the design of theprobe. The raw color information may comprise point, line, or areasamples. The raw color information may be adjusted before mapping usingcalorimetric calibration and intensity calibration data. During themapping process, the color information may be adjusted for the probe topolygon distance at point of color capture and polygon orientation toprobe at point of capture. The basis for the adjustments is a set ofcalibration procedures carried out for each individual probe.

The viewpoint for the surface displayed can have a constant position,zoom, and orientation in the world coordinate system of the object suchthat, as the probe is moved, the surface displayed increases where thedata is captured. The viewpoint is set before scanning starts, eitherwith an input device (such as buttons) on the arm, foot pedals, a mouse,and a keyboard, or by using the probe to determine the viewpoint.Alternatively, the viewpoint can have a constant position, zoom, andorientation in the probe coordinate system such that, as the probemoves, the surface is completely re-rendered at regular intervals, eachtime with the new surface displayed where the data has been captured,with the regular intervals being at an acceptable real-time rate, suchas 25 displays per second or less often. Alternatively, the viewpointcan have a constant position, zoom, and orientation in the worldcoordinate system where the surface displayed increases where the datais captured that is completely updated to that of the probe coordinatesystem on operator demand such as by the depressing of a button or footpedal or at regular time intervals, such as every 10 seconds. Thedifferent methods for updating the viewpoint provide differentadvantages, depending on the size and type of the object being scannedand the speed of the computer in recalculating the surface display froma different viewpoint.

Referring again to FIG. 2, the display 7 can be mounted on the probe 3such that the rendered surface 13, or other image displayed, moves withthe probe movement. Referring now to FIG. 21, the display 7 could bemounted in front of the operator's eyes as part of a heads-up display271, which may or may not also allow the operator 270 to see his realenvironment as well as the rendered surface 13, or it can be mountedelsewhere. In practical use, it is found that the operator watches therendered surface 13 on the display 7 while scanning because this has theimportant advantage of ensuring that all the object is scanned. In someapplications, such as scanning large objects or using a turntable torotate the object, watching the display 7 as a large display monitorsituated on a workbench can be advantageous. In other applications, suchas scanning a spherical type object, a display screen on the probe 3 isadvantageous because the operator moves with the probe. With sufficientquality in a heads-up display and with no negative effects such asfeeling sick, a heads-up display may be best for nearly all applicationsbecause it is feeding back most directly to the operator.

Referring again to FIG. 2, it is already technically possible tointegrate the display 7 with the probe 3 as, for instance, a color LCDscreen is small, lightweight, realtime and flat, while having sufficientresolution to render the surface so that the operator can see what hasand what has not been scanned. A display mounted on the probe could betiltable in one or two axes relative to the probe. The ability to tiltthe display relative to the probe can give the operator improved abilityto scan in spaces with poor visual access. Buttons 6 on the probe can beused to navigate menus and select from menus.

As computing power becomes faster and more compact, it will be possibleto encapsulate the computer 4 in the probe 3 as well as having thedisplay 7 mounted on the probe. The probe might have memory 262, whichcould be both dynamic memory and magnetic memory, such as a CDROM ordigital video disk (DVD). The probe might have a local power source 260,such as batteries. This would be the case with one or more remoteposition sensors 261 mounted inside the probe. Although one remoteposition sensor is sufficient, more accuracy is obtained by averagingthe positions coming from three or more remote position sensors. Anotherbenefit of three or more sensors is that when a spurious position isoutput by one or more sensors, this can be detected and the dataignored. Detection of incorrect positions is by means of comparing thepositions output by the three sensors to their physical locations withinthe probe to see if the variation is larger than the combined,acceptable error of the sensors. Since remote position sensor technologyis likely to remain much less accurate than multiply jointed armtechnology, it is preferable that probes with remote sensors use arrayscanning means rather than stripe scanning means. With a single arrayscan, all the data in the array (i.e., a range image) is accuratelyregistered to each other, but with stripes there are position errorsbeen any two sequential stripes. It is possible to use an iterativeclosest point (ICP) algorithm on overlapping range images tosubstantially reduce the errors caused by the remote position sensors;but this is not possible with stripes.

A number of different technologies exist for area probes includingbinary stereo, photometric stereo, texture gradients, range from focus,range from motion, time of flight, Moire interferometric, and patternedstructured light systems. The most common systems in use in industrialapplications are time of flight, Moire, and patterned structured light.Different area probe technologies have different advantages anddisadvantages for manual scanning.

Time of flight systems use a modulated laser spot to measure a scene bythe phase shift between outgoing and reflected beams, which isproportional to the range of the object point. A complete range image iscaptured by scanning the whole region of interest. For a small area,this technique is advantageous since it is line of sight, although theaccuracy is generally of the order of 1-2 mm unless multiplemeasurements are taken at each point, thus reducing scanning speedsignificantly. It is thus too slow.

Moire systems use gratings in front of projection and viewing optics toproduce an interference pattern that varies according to local changesin height on the object. Absolute measurements and measurements acrossdiscontinuities are only possible by taking several measurements withdifferent grating configurations or from different project angles. Forrelative height measurement, these systems offer high accuracy. It isthus too problematic to obtain absolute measurements.

A depth from focus range area sensor has recently been demonstrated thatallows the real-time determination of range from pairs of single imagesfrom synchronized cameras, albeit with the use of relatively complexhardware. It is thus too complex to use at this point in the developmentof the technology.

Referring now to FIG. 29(b), patterned structured light systems come inmany families and rely on projection of a light pattern and viewingoff-axis from the projection angle. Synchronously scanned lasertriangulation probes can be raster scanned over a 2D area. A laserstripe triangulation line can be scanned in one direction to producearea measurements. Scanning can be mechanical or electronic. Multiplelaser or light stripes 363 can also be simultaneously projected over anobject to obtain the same effect as a scanned stripe, but this has thedisadvantage that, in a single image it is not possible to differentiatebetween the stripes. To overcome this problem, a number of systems use agray-coded sequence of binary stripe patterns that solves the ambiguityproblem. However, the sensor should remain stationary during the captureprocess. An alternative solution is the projection of color-coded lightstripes that allow the unambiguous determination of range, even withdepth discontinuities from a single image. Note that the simultaneoususe of a number of stripes is herein classified as an area technique andnot a stripe technique.

The simultaneous projection of color-coded light stripes overcomes thedisadvantages of the previously described systems and is the preferredarea embodiment of this invention. Each stripe is one color. Each colormay be a discrete wavelength, such as provided by a number of differentlaser diodes or a subset of a spectrum range of color generated from awhite light source. Either all of the colors may be unique or a smallnumber of colors may repeat. The repetition of a small number of colorscan lead to ambiguity if stripes of the same colors are not sufficientlyseparated.

The probe encapsulation would have advantages in terms of cost reductionand complete flexibility in freedom of use because even cables may notbe required and the only limits would be the range and accuracy of theremote position sensor.

If an arm is being used as the position sensor, the probe with a displaymounted on it might receive its power along a cable that may follow thepath of the arm, and the computer may be situated in the base of thearm, which would reduce the weight of the probe and reduce operatorfatigue.

Referring again to FIG. 21, if a heads-up display 271 is being used, theprobe 3 with one or more remote position sensors 261 could be mounted onthe operator's head 270 with fixing means 272 to produce a head-mountedscanning system 274. This would lead to hands-free scanning, althoughsome method of navigating menus, e.g., verbally with speech recognitionby means of a microphone 273 or via buttons would be important inpractice. It is likely that the standoff from the probe to the objectusing a head-mounted scanning system 274 would be quite large, forexample, 250 mm, but it could be more or less.

There are several ways of automatically polygonizing intermediate datato form a 3D polygonal model. Two ways are described—strippolygonization and range image polygonization.

The strip polygonization of intermediate data to automatically create apolygonal model is described for a stripe scanner. The followingdescription is by means of an example and comprises the following steps:

1. Take the intermediate data in the order in which it is scanned,including the probe orientation for each stripe. For a stripe probe,this will typically consist of a number of neighboring stripes withoccasional discontinuities, such as when the scanning process is pausedor a turntable is turned or the direction of scanning is reversed. Theintermediate data is preferably in an encoded stripe form as describedabove.

2. Group the data into stripes of similar probe orientations and nodiscontinuities. An acceptable variation of the probe orientation in agroup of data may be ten degrees. The average normal for each set ofstripes is specified. A new group is started each time a discontinuityappears or when the probe orientation varies unacceptably.

3. If not already done in the intermediate data, filter the stripes ineach group using a chordal tolerancing routine to reduce the quantity ofpoints and maintain the positions of the break and jump flags.

4. Use a 2.5D polygonization method to polygonize each group. This willresult in a number of 2.5D polygonal meshes. There may be holes in anyof the meshes. The method eliminates occluded surfaces behind surfacesresulting from variations in the probe orientation within the group.

5. Use a polygon mesh integration method such as an implicit surfacemethod to integrate the 2.5D polygonal meshes into a computer modelcomprising one or more 3D polygonal meshes.

6. If required, use the known base plane of the object specified duringthe scanning setup to automatically close the bottom of the model wherethe object could not be scanned because it was resting on a table orturntable.

7. If required, use a general closing function to automatically closeall holes in the model.

8. If required, use a smoothing function set such that features createdby known levels of inaccuracy in the 3D scanning process are smoothedout and features greater in size than the inaccuracy of the system aremaintained.

9. Convert the internal polygon format into an output file of a commonlyused polygon file format, such as DXF.

The range image polygonization of intermediate data to automaticallycreate a polygonal model is similar to strip polygonization. Each rangeimage is effectively a group of stripes with the same surface normal.Steps 1 and 2 above are, therefore, not needed. There are two ways ofcarrying out the equivalent of step 3 above. Range image data may bechordal toleranced as a series of stripes as described in step 3, andthe polygonization process continued with steps 4 to 9, as required. Inthe second way, given the greater structure of a range image over agroup of stripes, steps 3 and 4 may be combined and a range imagetolerancing algorithm combined with a 2.5D polygonization algorithm andthe polygonization process continued with steps 5 to 9, as required.

Area scanners usually output range images. In general, range imagepolygonization is better suited to area scanners and strippolygonization is better suited to stripe scanners. If the intermediatedata structure is range images then the range image polygonization willwork whether each range image relates to a particular data captureinstant or is part of a defined range image structure that ischaracterized by the shape of the object.

The combining of color data onto the 3D model is known as texturemapping.

Before raw color data in the form of color images can be texture mappedonto the 3D model, it must first be corrected by means of variouscalibrations.

An important calibration is the geometric calibration of the colorcamera and finding the alignment transform of the color camera to thecalibrated 3D measurement system in the probe. Without thesecalibrations/alignments, neighboring color samples when mapped togetherwill produce visible errors. The objective of these calibrations is toget the geometric errors much smaller than those of the arm accuracy.The first geometric calibration is to take out lens distortion. Standardmeans are used for this based on imaging geometric objects of known sizeand extracting pixel coordinates using standard image processingtechniques. The second is to create the camera model. A simple pinholemodel can be used or a more complex model. Standard means are used forthis based on imaging geometric objects of known size from differentdistances and extracting pixel coordinates using standard imageprocessing techniques. The third is generating the alignment transform.A method has been developed based on 3D and color imaging geometricobjects of known size using the probe. For all three methods, athree-axis computer controlled machine is used to ensure precisedistances. The probe engineering must be geometrically stable enoughsuch that this transform will only be recalculated rarely such as afterthe probe has been dropped or damaged.

Much of the effect of distance from the probe to the object on recordedlight intensity can be calibrated out. A diffuse, flat, white surface isimaged normal to the camera axis at a number of different distances fromthe probe to the surface. The distances are chosen to cover the wholescanning range from closest point to furthest point. The variations inmean intensity recorded in the camera are used to calibrate the probewith distance. This calibration data is used to correct the color datarecorded when scanning an object such that all color data is correctedto a known distance equivalent.

Much of the effect of tilt of the surface from the camera axis on thecolor quality can be removed, but the effectiveness of this depends onat least the surface reflectance for each color. A diffuse, flat, whitesurface is imaged at various angles to the camera axis at a fixeddistance from the probe to the surface. The angles are chosen to thepoint at which there is significant deviation from the Lambertian model.The variations in mean intensity recorded in the camera are used tocalibrate the probe intensity with relative surface angle to the probe.This calibration data is used to correct the color data recorded whenscanning an object such that all color data is corrected to a normalequivalent.

A standard calorimetric calibration is carried out using referencecolors, such as Macbeth charts that are mounted normal to the colorcamera axis at a known distance from the probe. Corrections are made toa commonly used color standard, such as to the CIE. Individual pixels inthe camera may be color- and intensity-corrected.

Some of the above calibrations vary little among probes manufactured tothe same design. This is probably due to tight manufacturing tolerances.The calibration information can be incorporated into the software as,for example, constants or tables or equations for the probe design.Others calibrations are carried out once on the setup of each probeafter manufacture. Other calibrations could be carried out each time thescanning system is used—for example, the scanning of a white surface ata known distance will set the lamp intensity relative to the intensitywhen the bulbs were new.

Referring now to FIG. 22, there are several methods of mapping colorimages 320 onto the 3D model 324 to form texture maps. Surface elementson the 3D model may be flat polygons or elements of a high-level surfaceform. A mapping method for color images is:

1. Each color image 320 is corrected using calibration and geometricdata.

2. For each surface element 321, the color image whose normal 323 isclosest in orientation to the normal 322 of the surface element 321 isselected (the master image) and the texture map coordinates for thatsurface element go to the mapping of that surface element onto thatmaster image. The closest image normal is that of 320a in this case.

3. The other color images that map onto the surface element are thenprocessed. If the surface normal difference between the surface elementand a color image is above a certain tolerance, then that image isignored. This is because the color quality obtained in the imagedegrades significantly as the surface orientation of the object relativeto the image becomes very steep. The part of the master image on whichthe surface element maps is then improved by a weighted average of allthe color image mapped parts. The basis of the weighting is the cosineof the difference in surface normal between the surface element and thecolor image.

The apparatus and methods disclosed above each singly produce animproved color “copy” of the 3D model and a significant commercialadvantage.

Ways of improving the scanning timing and consequently reducinggeometrical errors are disclosed.

Where no electrical triggering is possible, to reduce the inaccuracycaused by the time difference between the recording of the arm positionand the capturing of the frame, the following method is employed:

1. With reference now to FIG. 23, the arm position before the frame iscaptured B is recorded and the time t1 of this is recorded.

2. A frame is requested.

3. When the frame has been captured C, the time t2 is recorded. There isa known delay T/2 with little variability from the middle of the framecapture to this time t2, which is largely dependent on the shutter timeopen T.

4. The arm position after A is recorded and the time t3 of this isrecorded.

5. The arm position in the middle of the frame is estimated byinterpolating in six degrees of freedom between the two arm positionsB,A using the time (t2-T/2) at the middle of the frame capture as theinterpolation weighting between t1 and t3.

6. In the case of a long interrupt, if the difference between t1 and t3is significantly large, then the data is deleted.

This interpolation method can increase the accuracy of a non-triggeredsystem by a large amount and is extremely significant in the quest toobtain geometrically accurate data.

In addition, the operating system under which the interpolation softwareruns may be set to prioritize the interpolation software as highpriority so that the introduction of delays due to other software beingexecuted is minimized. Even if another software function interrupts thisprocess, the validity of the process is not impaired unless theinterrupting process is of extraordinarily long duration. Prioritizationis not essential, but will contribute to reduced timing error whereprioritizing is available in the operating system.

In the case where triggering is possible, there are many methods ofcarrying it out. One method is, with reference now to FIG. 24, that thesynchronization signal 240 from a CCD camera 25 is stripped off byelectronic circuitry 241, and a relay 242 is used to generate a seriesof trigger pulses 243 to the arm computer 2. This has the advantage ofeliminating both the arm and camera variabilities and increasing theaccuracy of the scanning as much as possible for a given arm and camera.

The operator interface means alone—not including the standard computermeans such as mouse and keyboard—can be used to control the scanning andcomputer model generation process and the functionality of the optionsthat can be actuated. The operator interface means include means fornavigating menus such as buttons, foot pedals, joysticks, trackballs,and the position-sensing means—arm or remote position sensor.

Using any of the above means, the operator can simply select therequired operations and operating parameters, which could include, forexample, being able to:

-   -   Setup Scanning Apparatus    -   Select which position-sensing device is being used, i.e., arm.    -   Align the probe to the position-sensing device; align the        turntable.    -   Set the sampling of the points, i.e., sampling step or chordal        tolerance    -   Set when data thrown away because arm is moving too fast;    -   Data Collection    -   Pre-scan object to find out where it is.    -   Collect data points continuously, while that option is selected,        for example.    -   Collect one set of points such as a stripe section.    -   Collect sets of data points at pre-determined intervals of        position.    -   Collect contact reference points.    -   Pause and re-start data collection.    -   Collect Color Images.    -   Process    -   Generate polygonal or surface model from intermediate data.    -   Generate model in selected output format, e.g., 3DS, OBJ.    -   Map color images onto model.    -   Blend overlapping color images on model.    -   Close holes in polygon mesh.    -   Slice polygon mesh.    -   Smooth polygon mesh.    -   Decimate polygon mesh.    -   Flip normals in polygon mesh.    -   Change datum and orientation of coordinate system.    -   Edit    -   select/cut/paste/delete points    -   select/cut/paste/delete polygons    -   select/cut/paste/delete color images    -   Test    -   Check the performance of the system by processing data from        scanning a sphere.    -   Check the performance of the system by processing data from        scanning a flat surface.    -   Display    -   Display points in rendered color according to depth.    -   Redraw the computer display from the position and orientation of        the probe.    -   Select the field of view of the redraw, i.e., from zoom to wide        angle.    -   Select a viewpoint from list of preset viewpoints.    -   Display rendered data in one color.    -   Display rendered data using the scanned color data.    -   Display the computer model generated from polygons or complex        surfaces.    -   Model Data    -   Save the points/intermediate data/model onto a storage medium        such as a hard disk.    -   Publish the intermediate data/computer model as an object, such        as an object that may be automatically available to another        software package on the computer or over a network.    -   Load the points/intermediate data model from a storage medium,        such as a hard disk.    -   Range Image    -   Create a new linear range image using the position and        orientation of the probe when the option is selected.    -   Create a new cylindrical range image using the position and        orientation of the probe when the option is selected.    -   Select one of the defined range images from all the defined        range images.    -   Change the density of that range image.    -   Delete the selected range images.    -   Delete all range images.    -   Select a set of range images from a library of range image sets.        Library range image sets could be mathematically organized,        e.g., precisely orthogonal to each other, which may have        advantages in some uses of the scanned data.    -   Add the selected library set to the currently defined range        images.    -   Creating a new library set from the existing combination of        range images. In this way, if several similar objects are to be        scanned, the optimum range image combination can be set for the        first one and automatically reused on the others.    -   Setting the selected library set as the default library set. In        this way, for instance, a default library set of six range        images that form a cube may be used for many objects, such that        the process of range image definition is not needed, making the        total scanning process quicker.    -   Delete all current data points in all range images.    -   Delete all the data points in the selected range image only.    -   Display points from the selected range image only.    -   Display points from all range images.    -   Display points with different colors for each range image.    -   Update all range images from all the other range images by a        process of trying to fill in gaps or check entries in one range        image by studying the others.    -   Update the selected range image from all the other range images        by an inference process of trying to fill in gaps or check        entries in one range image by studying the others. This is        particularly useful when a new range image is defined after a        lot of scanning has taken place.    -   Constrain the size of a range image. This is often done when a        range image is defined specially to capture a small part of the        surface of the object that is not covered by the other range        images. This can be done to save memory on a computer with        limited memory and can also speed up the whole process    -   Choose and initiate an algorithm for automatically constructing        a model of polygons or complex surfaces from one range image.    -   Choose and initiate an algorithm for automatically constructing        a model of polygons or complex surfaces from all the range        images.    -   Set the parameters, such as the degree of accuracy, by which an        algorithm constructs a model of polygons or complex surfaces.    -   Select an integration algorithm that combines the polygon models        that have been generated from the range images.    -   Select a predefined sequence of algorithms that automatically        generates a complete model of polygons or complex surfaces from        a set of range images.

Complex surfaces can be created from marked surface patch boundaries.Referring now to FIG. 25, the object 130 is painted a uniform color (ifnecessary) before marking the patch boundaries 131 by hand in anothercolor, e.g., using a black marker pen on a white object. It is notimportant to mark these boundaries accurately, as they usually lie awayfrom features such as edges or rapid changes in surface normal. Theobject is then scanned in using one of the methods disclosed. The colorinformation is then used to automatically generate the patch boundariesby means of an algorithm that separates out the points 132 lying on thepatch boundaries by means of a color filter and then fits patch boundarylines such as splines 133 to these points. The edges may also bedetected using a separate algorithm. The patch boundaries that have beenautomatically created from the scan can then be used to create thecomplex surface model. The main benefit of this method is that it iseasier to mark patch boundaries on the object than on the computer modelprior to the automatic creation of the complex surface model. Referringnow to FIG. 26(a), an important implementation 333 of the invention isdisclosed in which the multiply-jointed arm 1 is mounted on the end ofthe horizontal arm of the horizontal arm measuring machine 330 forscanning a large object 331. The horizontal arm measuring machine 330has a machine control box 332 that outputs the position of the machineto the computer 4. The arm control 2 and the probe 3 are also connectedto the computer 4. This implementation makes the scanning of largeobjects more precise in that either a large arm or leapfrogging would beless accurate than a horizontal arm, and simpler in that each time thehorizontal arm is moved, the software takes it into accountautomatically rather than needing to reregister using a leapfroggingmethod. In industry, firms that have large objects, such as automotivemanufacturers, usually have horizontal arm machines so this makes theimplementation particularly attractive.

Referring now to FIG. 26(b), firms that have large objects, such asautomotive manufacturers, often have two horizontal arm machinessituated opposing each other, both of which can reference to the sameobject coordinate system. In this case, the whole of the object may bescanned by scanning part of the object with the probe fitted to thefirst horizontal arm machine and the rest of the object with the probefitted to the second horizontal arm machine.

This invention is a general 3D model-making device and has wide-rangingapplicability. The application industries for this invention includedesign stylists who need to turn clay objects into computer modelsquickly and accurately; games developers and animators who need toconvert new characters into 3D data sets for animation; shoemanufacturers who need to make custom shoes; automotive manufacturerswho need to model the actual cable and pipe runs in confined spaces; andmedical applications that include radiotherapy and wound treatment.Altogether, some 200 applications have been identified for thisinvention.

Referring now to FIG. 27, as an example of the applications for thescanning apparatus 100 in accordance with the invention, the scanningapparatus 100 can be used to scan a human foot 141 with full body weighton it on surfaces of different resilience is also disclosed. The outsideof the foot 141 is first scanned using the methods and devices disclosedabove with the required amount of body weight being exerted. The foot141 is then removed and a second scan is carried out of the surface 142on which the foot 141 was pressed. The first scan is a positive. Thesecond scan is a negative. The surface normals of the second scan arethen reversed by means of a simple algorithm and the two scans combinedto give the positive shape of the foot. It is important that if adeformable material is used that it does not spring back. Such amaterial might be sand, clay, or plaster. Materials of differentresilience may be appropriate for different applications. This method isalso appropriate when the foot is pressed onto the lower half of a shoewith the sides cut away.

There is a need by automobile manufacturers to identify the actual routeof pipes and cables in confined areas, such as an engine department.Automobile manufacturers are trying to model in 3D CAD all aspects of acar. They need some way of scanning pipes and cables in the carreference system so that high level 3D models of the pipes and cablesare output that can be introduced into the CAD system for identifyingactual routing and potential interferences. In the scanning of pipes andcables, for instance, in confined spaces, if there is a problem withblack or shiny items not being scannable, these can be first dusted witha white powder that is easily removed after scanning.

Referring now to FIG. 28(a), it is often better to scan a cable or pipe341 as a number of stripe sections 342 to 349, rather than as a largenumber of densely spaced stripes. A stripe sensor can be activated in afirst mode to take a single stripe section by the operator activating abutton or foot-pedal. In this way, the operator can take a small numberof sections to describe the path of the pipe using his expertise todecide when to take sections. For instance, where a pipe joins anotherpipe, it may be appropriate to capture many more stripe sections 344 to349. Also, where there is a feature such as a fixing on a pipe, it maybe appropriate to capture very dense stripes. A second mode would becapturing stripe sections as fast as the sensor can capture them anddisplaying them as a surface on the display. A third mode would be amode in which the operator specifies the distance between the sections,e.g., 5 mm, and the system automatically takes a stripe section every,e.g., 5 mm that the stripe travels in 3D space. One method ofdetermining this distance is to select the point at the average standoffdistance in the middle of the stripe, i.e., the center point of themeasuring range, and when this point has moved 5 mm, to automaticallycapture another stripe section. When the operator is scanning pipes andcables, the operator control system should support the simple switchingbetween the three modes.

The intermediate data structure in which the stripe sections arecollated could be the standard stripe section structure 303, butincludes the changes in mode and the orientation of the probe for eachsection. In scanning pipes and cables, panel sections along which thepipes and cables run are also captured 342a, 342d. Where there is nocontact between the pipe and the panel, there is a jump or break in thestripe section. These can be flagged in the data structure with jumpflags 305 and break flags 304.

To be useful to an automobile manufacturer, a high level model should becreated and output from this data. A polygonization or surfacing methodjoins the sections together and can handle the joining of pipes, panels,etc. The result is high level models 350 to 352. If more information isknown about the pipe or cable, such as its section if it is constant orits form even if the form's dimensions change, e.g., circular butvarying diameter, the model 351 can be automatically expanded to 353.Alternatively, two scanned sides of the same pipe can be automaticallyjoined. This gives the automobile manufacturer the high level model thathe needs.

As will be understood to persons skilled in the art, there are variousmodifications within the scope of the present invention. For example,the color camera does not need to be included. A single camera could beutilized for both color and position sensing. The fitter in the probecould be a narrow band pass filter or a red high band pass filter, asrequired. The system is adaptable to many types of model generation notjust those discussed herein. The data collected by the probe could beused for other applications and could be stored for disseminationelsewhere—for example, by electronic mail. The probe can be a stripe oran area probe. The display can be mounted anywhere depending upon theapplication requirements.

While the preferred embodiment of the invention has been illustrated anddescribed, it will be appreciated that various changes can be madetherein without departing from the spirit and scope of the invention.

1. A scanning apparatus, comprising: a multiply-jointed arm having aplurality of arm segments and a data communication link to transmitdata; and a scanner mounted on an arm segment of the multiply-jointedarm for movement therewith to capture data from a plurality of points ona surface of an object, the scanner having a housing enclosing: (a) alight source operable to emit light onto the object surface; (b) a lightdetector operable to detect light reflected from the object surface andto generate electrical image data signals in dependence upon thedetected light; and (c) a data processor operable to process theelectrical image data signals to generate processed data of reducedquantity, the data processor being connected to the data communicationlink to transmit the processed data therealong.
 2. A scanning apparatusaccording to claim 1, wherein the data processor is operable to generatethe processed data of reduced quantity by processing the electricalimage data signals to generate measurement data and processing themeasurement data to reduce the quantity thereof.
 3. A scanning apparatusaccording to claim 1, wherein the data processor is operable to generatethe processed data of reduced quantity by filtering the data.
 4. Ascanning apparatus according to claim 1, wherein the data processor isoperable to generate the processed data of reduced quantity bydiscarding data.
 5. A scanning apparatus according to claim 1, whereinthe communication link comprises a cable.
 6. A scanning apparatusaccording to claim 1, further comprising a battery power supply withinthe apparatus to power the scanner.
 7. A scanner mountable on amultiply-jointed arm for movement therewith to capture data from aplurality of points on a surface of an object, the scanner having ahousing enclosing: a light source operable to emit light onto the objectsurface; a light detector operable to detect light reflected from theobject surface and to generate electrical image data signals independence upon the detected light; and a data processor operable toprocess the electrical image data signals to generate processed data ofreduced quantity, the data processor being connectable to a datacommunication link to transmit the processed data therealong.
 8. Ascanner according to claim 7, wherein the data processor is operable togenerate the processed data of reduced quantity by processing theelectrical image data signals to generate measurement data andprocessing the measurement data to reduce the quantity thereof.
 9. Ascanner according to claim 7, wherein the data processor is operable togenerate the processed data of reduced quantity by filtering the data.10. A scanner according to claim 7, wherein the data processor isoperable to generate the processed data of reduced quantity bydiscarding data.
 11. A scanning apparatus, comprising: amultiply-jointed arm having a plurality of arm segments; a scannermounted on an arm segment of the multiply-jointed arm for movementtherewith to capture data from a plurality of points on a surface of anobject, the scanner having a housing enclosing: (a) a light sourceoperable to emit light onto the object surface; (b) a light detectoroperable to detect light reflected from the object surface and togenerate electrical image data signals in dependence upon the detectedlight; and (c) a data processor operable to process the electrical imagedata signals to generate digital image data; and a bus connected to thedata processor of the scanner to transmit the digital image data.
 12. Ascanning apparatus according to claim 11, wherein the data processorcomprises a frame grabber.
 13. A scanning apparatus according to claim11, further comprising a battery power supply within the apparatus topower the scanner.
 14. A scanner mountable on a multiply-jointed arm formovement therewith to capture data from a plurality of points on asurface of an object, the scanner having a housing enclosing: a lightsource operable to emit light onto the object surface; a light detectoroperable to detect light reflected from the object surface and togenerate electrical image data signals in dependence upon the detectedlight; and a data processor operable to process the electrical imagedata signals to generate digital image data, the data processor beingconnectable to a bus to transmit the digital image data.
 15. A scanneraccording to claim 14, wherein the data processor comprises a framegrabber.
 16. A coordinate measuring machine, comprising: amultiply-jointed arm having a plurality of arm segments and a physicaldata path to transmit data; and a scanner mounted on an arm segment ofthe multiply-jointed arm for movement therewith to capture data from aplurality of points on a surface of an object, the scanner having ahousing enclosing: a light source operable to emit light onto the objectsurface; a light detector operable to detect light reflected from theobject surface and to generate electrical image data signals independence upon the detected light; and a data processor operable toprocess the electrical image data signals to generate data definingcoordinate measurements of the surface of the object, and to transmitthe generated data on the physical data path.
 17. A coordinate measuringmachine according to claim 16, wherein the data processor is arranged toprocess the electrical image data signals to generate data definingcoordinate measurements comprising three-dimensional positions.
 18. Acoordinate measuring machine according to claim 16, wherein the dataprocessor is arranged to process the electrical image data signals togenerate data defining coordinate measurements comprising points inthree-dimensional space.
 19. A coordinate measuring machine according toclaim 16, wherein the data processor is arranged to process theelectrical image data signals to generate data defining coordinatemeasurements comprising connected polygons in three-dimensional space.20. A coordinate measuring machine according to claim 16, wherein thephysical data path comprises a cable.
 21. A coordinate measuring machineaccording to claim 16, further comprising a batter battery power supplywithin the apparatus to power the scanner.
 22. A scanner mountable on amultiply-jointed arm for movement therewith to capture data from aplurality of points on a surface of an object, the scanner having ahousing enclosing: a light source operable to emit light onto the objectsurface; a light detector operable to detect light reflected from theobject surface and to generate electrical image data signals independence upon the detected light; and a data processor operable toprocess the electrical image data signals to generate data definingcoordinate measurements of the surface of the object, and to transmitthe generated data on a physical data path.
 23. A scanner according toclaim 22, wherein the data processor is arranged to process theelectrical image data signals to generate data defining coordinatemeasurements comprising three-dimensional positions.
 24. A scanneraccording to claim 22, wherein the data processor is arranged to processthe electrical image data signals to generate data defining coordinatemeasurements comprising points in three-dimensional space.
 25. A scanneraccording to claim 22, wherein the data processor is arranged to processthe electrical image data signals to generate data defining coordinatemeasurements comprising connected polygons in three-dimensional space.26. A laser scanning apparatus, comprising: a multiply-jointed armhaving a plurality of arm segments and a data communication link totransmit data; and a laser scanner mounted on an arm segment of themultiply-jointed arm for movement therewith to capture data from aplurality of points on a surface of an object, the laser scanner havinga housing enclosing: (a) a laser to emit a laser stripe onto the objectsurface; (b) a camera operable to generate images of laser lightreflected from the object surface; and (c) a data processor operable toprocess the images generated by the camera to generate processed datadefining a position of the laser stripe in the images, the dataprocessor being connected to the data communication link to transmit theprocessed data therealong.
 27. A laser scanning apparatus according toclaim 26, wherein: the camera is arranged to generate images comprisinga plurality of pixels; and the data processor is arranged to process theimages generated by the camera to generate processed data defining aposition of the laser stripe in the images to sub-pixel accuracy.
 28. Alaser scanning apparatus according to claim 26, wherein the datacommunication link comprises a cable.
 29. A laser scanning apparatusaccording to claim 26, further comprising a batter battery power supplywithin the apparatus to power the laser scanner.
 30. A laser scannermountable on a multiply-jointed arm for movement therewith to capturedata from a plurality of points on a surface of an object, the laserscanner having a housing enclosing: a laser to emit a laser stripe ontothe object surface; a camera operable to generate images of laser lightreflected from the object surface; and a data processor operable toprocess the images generated by the camera to generate processed datadefining a position of the laser stripe in the images, the dataprocessor being connectable to a data communication link to transmit theprocessed data therealong.
 31. A laser scanner according to claim 30,wherein: the camera is arranged to generate images comprising aplurality of pixels; and the data processor is arranged to process theimages generated by the camera to generate processed data defining aposition of the laser stripe in the images to sub-pixel accuracy.
 32. Alaser scanning apparatus, comprising: a multiply-jointed arm having aplurality of arm segments and a data communication link to transmitdata; and a laser scanner mounted on an arm segment of themultiply-jointed arm for movement therewith to capture data from aplurality of points on a surface of an object, the laser scanner havinga housing enclosing: (a) a laser to emit at least one laser stripe ontothe object surface; (b) a camera operable to generate images of laserlight reflected from the object surface, each image comprising aplurality of pixels; and (c) a data processor operable to process theimages generated by the camera to perform measurements to sub-pixelaccuracy, the data processor being connected to the data communicationlink to transmit results of the measurements therealong.
 33. A laserscanning apparatus according to claim 32, wherein the data communicationlink comprises a cable.
 34. A laser scanning apparatus according toclaim 32, further comprising a batter battery power supply within theapparatus to power the laser scanner.
 35. A laser scanner mountable on amultiply-jointed arm for movement therewith to capture data from aplurality of points on a surface of an object, the laser scanner havinga housing enclosing: a laser to emit at least one laser stripe onto theobject surface; a camera operable to generate images of laser lightreflected from the object surface, each image comprising a plurality ofpixels; and a data processor operable to process the images generated bythe camera to perform measurements to sub-pixel accuracy, the dataprocessor being connectable to a data communication link to transmitresults of the measurements therealong.
 36. An apparatus comprising: ascanner including a light source which emits light to an object so thatthe light reflects off the object, and a light detector which detectsthe reflected light, wherein the scanner outputs information indicatedby the detected light; a position detector detecting position of atleast one of the object and the scanner at a timing corresponding to atiming at which the light detector detects the reflected light; aprocessor determining a relative position of the object to the scannerusing the information output by the scanner; and a three dimensionaldata generator generating three-dimensional data of the object using thedetermined relative position and the information output by the positiondetector.
 37. An apparatus comprising according to claim 36, wherein thetiming at which the light detector detects the reflected light is atleast one of different timings defined by a synchronization signal; andthe apparatus further comprising a trigger pulse generator receiving thesynchronization signal and, in response thereto, outputting triggerpulses to the position detector, which indicate the timing at which theposition detector is to detect the position.
 38. An apparatus accordingto claim 37, wherein timing at which the trigger pulse generator outputstrigger pulses is a predetermined time behind timing at which thetrigger pulse generator receives the synchronization signal.
 39. Anapparatus comprising according to claim 36, wherein the positiondetector is a remote position sensor.
 40. An apparatus comprisingaccording to claim 36, wherein the position detector calculates theposition to thereby detect the position.
 41. An apparatus comprisingaccording to claim 36, further comprising: a probe which captures datafrom individual points on the object touched by the probe.
 42. Anapparatus comprising: a scanner including a light source which emitslight to an object so that the light reflects off the object, and alight detector which detects the reflected light, so that the scannerthereby scans the object, wherein the scanner outputs informationindicated by the detected light; and a processor which, as the object isbeing scanned by the scanner, determines a changing relative positionalrelationship between the object and the scanner at a timingcorresponding to a timing at which the light detector detects thereflected light, and a three dimensional data generator generatingthree-dimensional data of the object using the information output by thescanner and the determined change in positional relationship.
 43. Anapparatus comprising: a scanner including a light source which emitslight to an object so that the light reflects off the object, and alight detector which detects the reflected light, wherein the scanneroutputs information indicated by the detected light; a position detectordetecting position of at least one of the object and the scanner at atiming corresponding to a timing at which the light detector detects thereflected light; and a processor determining a relative position of theobject to the scanner using the information output by the scanner, andgenerating three-dimensional data of the object using the determinedrelative position and the information output by the position detector.44. An apparatus comprising: a scanner including a light source whichemits light to an object so that the light reflects off the object, anda light detector which detect the reflected light, wherein the scanneroutputs first information indicated by the detected light; a positiondetector which processes second information related to position of atleast one of the object and the scanner at a timing related to a timingat which the light detector detects the reflected light; a processorwhich communicates with the position detector and the scanner, and whichgenerates three-dimensional data of the object using the firstinformation outputted by the scanner and the second informationoutputted by the position detector.
 45. An apparatus according to claim44, wherein the timing at which the light detector detects the reflectedlight is at least one of different timings defined by a synchronizationsignal; and the apparatus further comprising a trigger pulse generatorreceiving the synchronization signal and, in response thereto,outputting trigger pulses, of which each indicates a timing at which theposition detector is to detect the position, to the position detector.46. An apparatus according to claim 45, wherein the timing at which thetrigger pulse generator outputs trigger pulses is a predetermined timebehind a timing at which the trigger pulse generator receives thesynchronization signal.
 47. An apparatus according to claim 46, whereinthe position detector calculates the position to thereby detect theposition.
 48. An apparatus comprising: a scanner including a lightsource which emits light to an object so that the light reflects off theobject, and a light detector which detects the reflected light, whereinthe scanner outputs first information indicated by the detected light; aprocessor unit which communicates with the scanner and which generatesthree-dimensional data of the object using the first informationoutputted by the scanner and second information relative to a positionof at least one of the object and the scanner at a timing correspondingto a timing at which the light detector detects the reflected light. 49.An apparatus according to claim 48 further comprising: a positiondetector which detects relative position between the scanner and theobject at the timing corresponding to the timing at which the lightdetector detects the reflected light.
 50. An apparatus according toclaim 48, wherein the timing at which the light detector detects thereflected light is at least one of different timings defined by asynchronization signal; and the apparatus further comprising a triggerpulse generator receiving the synchronization signal and, in responsethereto, outputting trigger pulses, of which each indicates a timing atwhich the position detector is to detect the position, to the positiondetector.
 51. An apparatus according to claim 49, wherein the timing atwhich the light detector detects the reflected light is at least one ofdifferent timings defined by a synchronization signal; and the apparatusfurther comprising a trigger pulse generator receiving thesynchronization signal and, in response thereto, outputting triggerpulses, of which each indicates a timing at which the position detectoris to detect the position, to the position detector.
 52. An apparatusaccording to claim 50, wherein a timing at which the trigger pulsegenerator outputs trigger pulses is a predetermined time behind a timingat which the trigger pulse generator receives the synchronizationsignal.
 53. An apparatus according to claim 51, wherein a timing atwhich the trigger pulse generator outputs trigger pulses is apredetermined time behind a timing at which the trigger pulse generatorreceives the synchronization signal.
 54. An apparatus according to claim52, wherein the position detector calculates the position to therebydetect the position.
 55. An apparatus according to claim 53, wherein theposition detector calculates the position to thereby detect theposition.
 56. A method for generating three-dimensional data of theobject, comprising: irradiating light onto the object from a scanner;detecting light reflected from a surface of the object by the scanner;outputting information indicated by the detected light from the scanner;detecting relative position of the object to the scanner at a timingrelated to a timing at which the reflected light is detected; andgenerating three-dimensional data of the object using the detectedrelative position and the outputted information.
 57. A method accordingto claim 56, further comprising: generating a synchronization signalfrom the scanner; and generating a trigger pulse, in response toreceiving the synchronization signal at a trigger pulse generator,wherein the relative position of the object to the scanner is detectedat a timing at which the trigger pulse is detected by the positionsensor.
 58. A method according to claim 57, wherein the trigger pulse isoutput at a timing which is a predetermined time behind a timing atwhich the trigger pulse generator receives the synchronization signal.59. A method for generating three-dimensional data of an object,comprising: irradiating light onto the object from a scanner; detectinglight reflected from a surface of the object by the scanner; outputtinginformation indicated by the detected light from the scanner; generatinga trigger pulse synchronized with a timing at which the sensor detectslight; processing position information relative to the position of thescanner; outputting the position information in response to the triggerpulse; and generating three-dimensional data of the object using theposition data and the information output from the sensor.